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Image segmentation generally concerns selection and/or separation of a 
selected part of a dataset. Such a dataset notably represents image information of an imaged 
object and the selected part relates to a specific part of the image. The dataset is in general a 
multi-dimensional dataset that assigns data values to positions in a multi-dimensional 
geometrical space. In particular, such datasets can be two-dimensional or three-dimensional 
images where the data values are pixel values, such as brightness values, grey values or color 
values, assigned to positions in a two-dimensional plane or a three-dimensional volume. 

The invention relates to a method of segmenting a selected region from a 
multi-dimensional dataset, the method comprising the steps of 

- setting-up a shape model representing the general outline of the selected region 

- setting-up an adaptive mesh representing an approximate contour of the selected region 

- which adaptive mesh is initialized on the basis of the shape model . 

Such a method of segmenting a selected region from a three-dimensional 
dataset is known from the paper 'An efficient 3D deformable model with a self-optimising 
mesh ' by A. J. Bulpitt and N.E. Efford in Image and Vision Computing 14(1996) pp.573-580. 

The known method operates on a multi-dimensional dataset in the form of a 
three-dimensional image. The known method employs a triangular mesh to represent a 
surface of the selected region. A so-called distance transform is used to initialize the adaptive 
mesh and when the mesh is close to its final solution, an image grey level gradient is used to 
drive the deformation of the mesh. 



It is an object of the invention to provide a method of segmenting a selected 
region from a dataset, which method is computationally fast, more robust and more accurate 
than the conventional method. 
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This object is achieved by the method of segmenting a selected region from a 
dataset wherein according to the invention the adaptive mesh is deformed in dependence on 
the shape model and on feature information of the selected region. 



The selection of the selected region may be effected by the initialization of the 
segmentation. In particular the initialization is done on the basis of the shape model. Such a 
shape model represents the selected region in terms of a parameterization of a priori shape 
information. This a priori shape information and the positioning of the shape model in the 
multi-dimensional dataset determine the region that is being selected. In particular, the shape 
model is an active shape model that allows for adaptation of its translation and orientation 
and pose, but also of shape parameters representing smaller details of the shape of the region 
that is represented by the active shape model. The adaptive mesh represents an approximate 
contour. The approximate contour represents an estimate of the boundary of the selected 
region relative to the remainder of the multi-dimensional dataset. The contour is in general a 
hypersurface in the multi-dimensional dataset, such as a surface or a curve. The contour has a 
dimension that is smaller than the dimensionality of the multi-dimensional dataset As the 
adaptive mesh is deformed, the approximate contour more accurately approaches the 
boundary of the selected region. According to the invention the adaptive mesh is deformed 
on the basis of feature information together with information from the shape model. The 
feature information, for example, concerns image features such as edges and surfaces of 
approximately equal data values. Such feature information is preferably represented by local 
gradients in the dataset, i.e. local relative changes of the data values. Strong gradients 
indicate the presence of boundaries and the direction of the gradients is locally perpendicular 
to the boundary. The shape model is employed during the successive steps of adaptation of 
the mesh. For example, the adaptive mesh is deformed on the basis of an internal energy and 
according to the invention the internal energy is defined in dependence on the shape model. 
Thus, information provided by the shape model is more effectively employed to drive the 
adaptive mesh to the actual boundary of the selected region. Notably image features in the 
multi-dimensional dataset which would drive the adaptive mesh to false boundaries, so-called 
false attractors, are substantially avoided, while a particularly close initialization is not 
required. 
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These and other aspects of the invention will be further elaborated with 
reference to the preferred implementations and embodiments as defined in the dependent 
claims. 

Preferably, the shape model is updated upon deformation of the adaptive 
5 mesh. In particular the parameters of the shape model, such as the position, scale and pose, 
are adapted as the boundary is approached by the adaptive mesh. 

In a further preferred implementation of the invention one or several local 
surface patches of the selected region are detected. An individual local surface patch is a 
M small area tangent to the boundary of the selected region. In practice the local surface patch 
g 1 0 is taken as a small region transversely of to a local gradient in the neighbourhood of the 
J* 1 ' current adaptive mesh. During mesh adaptation, detection takes place of boundaries which 
O are in agreement with the adaptive mesh it is not considered whether they belong to the 
Jj4 object to be segmented. Normally, some boundaries are correctly detected and others are 
* misleading in the first few iterations of mesh adaptation. As the correct boundaries are more 

|y 15 consistent with the model than the misleading boundaries, the fraction of correctly detected 
fil boundary points increases during mesh adaptation. 

O Subsequently, an individual vertex of the adaptive mesh is moved in the 

fli 

direction transversely of to the local surface patch at issue so as to deform the mesh. In this 
way it is avoided that the mesh is deformed in the direction of 'false attractors'. 

20 Preferably, the deformation of the mesh is carried out in that individual 

vertices of the adaptive mesh are moved towards the surface patch in dependence on the 
angle between the local normal to the adaptive mesh and the local normal to the surface 
patch. The individual vertex is usually moved towards the surface patch that is closest to the 
vertex at issue. In this way the adaptive mesh is deformed while adequately taking into 

25 account available information on image features. Notably, the image features as represented 
by the orientation of the local surface patch is most relevant for deformation of the adaptive 
mesh while the avoiding influence of so-called false attractors. Particularly advantageous 
results are obtained when the vertices are moved essentially perpendicularly towards the local 
surface patches. 

30 Furthermore, gradients in the dataset that have a magnitude smaller than a pre- 

set threshold value are advantageously discarded in forming the surface patches and in 
deformation of the adaptive mesh. Thus, deviations in the adaptive mesh deformation due to 
false attractors are avoided even further. 
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A preferred implementation of the adaptive mesh deformation is based on sp- 
termed energy functions. The term 'energy' in this respect is not related to a physical energy, 
but is used because the concept of a deformable adaptive mesh shows strong resemblance 
with a string of masses (the vertices) interconnected by springs (the bonds) moving in under 
5 the influence of a force field and striving towards a stable situation of minimum energy. In 
this resemblance, the external energy pertains to the external force field applied to the 
vertices and the internal energy pertains to the interaction between the vertices. Notably, the 
internal energy relates to the shape of the adaptive mesh. According to the invention, 
. preferably the relative weight of the external energy relative to the internal energy is 

III 10 controlled by a parameter that may be adjusted by the user. This provides additional 
~y flexibility in implementing the deformation of the adaptive mesh in that the relative 

influences of the force fields and the shape of the adaptive mesh can be adapted. Preferably, 
*|* the external energy is made dependent on feature information of the selected region relative 
fl to the actual configuration of the adaptive mesh. Thus, it is achieved that the external forces 

&[ 1 5 drive the adaptive mesh towards image features, such as strong boundaries, of the selected 

|!| region. In order to control the shape of the adaptive mesh so as to resemble the shape of the 

(ft 

j»i selected region to a certain degree, the internal energy is advantageously dependent on the 
flJ shape model. Thus it is avoided that the external forces cause the adaptive mesh to strongly 
deviate from the raw shape of the selected region. The co-operation of the internal energy and 
20 the external energy leads to fast, reliable and accurate convergence of the adaptive mesh with 
the actual boundary of the selected region. As the shape model is updated upon deformation 
of the adaptive mesh, the co-operation of the internal energy and the external energy is 
further improved in that the most updated information is employed and deviations due to 
false attractors are effectively avoided. 
25 The invention also relates to a data processor for segmenting a selected region 

from a multi-dimensional dataset. The data processor according to the invention is defined in 
claim 9. The data processor according to the invention can to carry out the method according 
to the invention and notably avoids deviations due to false attractors. 

The invention also relates to a computer program which includes instructions 
30 for segmenting a selected region from a multi-dimensional dataset. The computer program 
according to the invention is defined in Claim 10. The computer program according to the 
invention is preferably loaded into the working memory of a data processor. The data 
processor is thus equipped to carry out the method of the invention. The computer program 
may be presented on a data carrier such as a CD-ROM. The computer program may also be 
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present viz. over a network such as the world-wide web and can be downloaded into the 
working memory of the data processor from such a network. 

These and other aspects of the invention are apparent from and will be 
elucidated with reference to the embodiments described hereinafter and with reference to the 
accompanying drawing. 

The sole Figure illustrates the deformation of the adaptive mesh as employed 
in the method of the invention. 

In a practical implementation of the method of the invention, a triangular 
adaptive mesh represents the deformable model. The adaptive mesh comprises N vertices 
with co-ordinates x x ,..Jc N . The adaptive mesh is adapted by way of an iterative procedure in 
which each iteration includes the following two steps: 

1. surface detection so as to detect local surface patches of the selected region, 

2. reconfiguration of the adaptive mesh so as to update the mesh in that the vertices of the 
mesh are moved towards the local surface patches. 

The reconfiguration of the mesh is done by minimizing the energy: 
E = E ext +ccE int 

The external energy E ext drives the adaptive mesh towards the surface patches 
obtained in the surface detection step. The internal energy E int restricts the flexibility of the 
adaptive mesh. The parameter a weights the relative influence of both terms. 

1 The individual steps will now be discussed in more detail. 
Surface detection is carried out for the triangle centres x l of the adaptive mesh. A search for 
surface patches is performed along the normal n l of the triangle so as to find the point xti 
having the optimum combination of feature value F(x) and distance j3 to the triangle centre 
x. : 



% =x l +Si l arg mm jpfS 2 - F(x t + y<5w,)] 
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Therein, the parameter D controls the combination of distance information and the feature 
value F(x), the parameter / defines the length of the profile searched for surface patches, and 
the parameter 8 is the distance between sample points on the profile. 

By combining feature value and distance information, surface points next to 
5 the adaptive mesh are given preference and the deformation of the adaptive mesh becomes 
much more stable as compared to detection of the point of maximum feature value. Thus, 
deviation due to false attractors is effectively avoided or reduced. 
For example, the quantity 

"J* is used as a feature value, where the sign is chosen in dependence on the brightness in the 
a selected region with respect to its surroundings. For surface points with a gradient magnitude 

|ij \Si I smaller than the threshold value g^, this feature value is essentially the gradient in the 

G 

$ direction of the local normal to the adaptive mesh. If the gradient value exceeds the threshold 
jl 5 value, the feature value represents the scalar product of the local normal to the mesh and the 
gradient direction. Thus, the feature value takes into account the angle between the gradient 
and the local normal to the mesh. 

Advantageously, the external energy has the form 



with the weights 

w, =max[o, F(x ( )-Dj 2 S 2 ] 

This external energy causes the vertices of the adaptive mesh to move perpendicularly to the 
local surface patch as shown in the Figure. 

A suitable choice of the internal energy is defined as follows. The starting 
point is a shape model represented by a mesh of triangles. The vertex co-ordinates of the 
mesh of the shape model are given by 
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In this expression ml > ,....mt 1 denote the vertex co-ordinates of the mean model, and mf, ... 
describe the variation of the vertex co-ordinates associated with the eigenmodes (k=l, ...M) 
of the model, p b .. p M denote the weights of the eigenmodes. 

The internal energy is conceived to maintain the distribution of vertices to 
some extent. Thus, pathological deformations due to false attractors are avoided. 
The internal energy is given as: 

e * = z z k -^k° + 5>*<# 

verticesi neighbours j [ |_ J=l,.M J J 

I?! where s and R represent the scale and orientation of the shape model. 
MO The deformation of the adaptive mesh takes place in two steps. In the first step 

jjp the scaling s and the orientation R of the shape model with the current weights p t of the 
41 eigenmodes are determined with respect to the current configuration of the adaptive mesh. 
CI Point-based registration methods based on singular value decomposition have been found to 
function appropriately in determining scaling and rotation. In the second step, the vertex co- 
if 11 5 ordinates xi and the weights p t are updated using the scaling s and the orientation R as 
O 

ry determined in the first step. The energy function £ is a quadratic function with respect to 
these parameters. Minimizing this energy is conveniently done by way of a conjugate 
gradient method which is known as such from the handbook 'Practical Optimisation' 
(Academic Press, San Diego 1981) by P.E. Gill et al. This method takes advantage of the fact 
20 that the matrix involved in the minimization is sparsely occupied. 

The deformation of the adaptive mesh until convergence to the accurate 
approximation of the boundary of the selected region takes only a short time, that is, up to 
about 30s, which makes the segmentation according to the invention quite practical even in 
the event that large datasets are involved. Notably, such large datasets may occur in medical 
25 diagnostic applications where three-dimensional datasets with a high spatial resolution are 
employed. Good results are obtained especially in the segmentation of vertebrae from three- 
dimensional images of the human spine. If no steps were taken, this application is quite prone 
to false attractors formed by parts of neighboring vertebrae. 



